[hadoop、Hbase错误解决合集] Hbase的Hmaster进程启动后自动关闭的问题解决方法 |
您所在的位置：网站首页 › hbase出现bad line › [hadoop、Hbase错误解决合集] Hbase的Hmaster进程启动后自动关闭的问题解决方法 |
[hadoop、Hbase错误解决合集] Hbase的Hmaster进程启动后自动关闭的问题解决方法
|
============================================================================== 现象 1.日志中提示： ERROR [Thread-14] master.HMaster: Failed to become active master 2.进入hbase shell中进行数据库操作提示： KeeperErrorCode = NoNode for /hbase/master 解决步骤： 一、首先打开hbase的日志文件，一般默认是在hbase的logs文件夹中 二、通过分析可知， ERROR [Thread-14] master.HMaster: Failed to become active master是导致Hmaster进程关闭的原因。 三、查看错误描述 java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of ‘hbase.procedure.store.wal.use.hsync’ to set the desired level of robustness and ensure the config value of ‘hbase.wal.dir’ points to a FileSystem mount that can provide it. 程序WAL（预写日志）依赖于hsync（行同步信号）能力在组件故障期间进行正确操作，但是底层文件系统不支持这样做。请检查’hbase.procedure.store.wal.use '的配置值。设置所需的健壮性级别，并确保配置值为’hbase.wal.dir’指向可以提供它的文件系统挂载。 关于hbase中hsync的简单介绍 hsync的语义是：client端所有的数据都发送到副本的每个datanode上，并且datanode上的每个副本都完成了posix中fsync的调用，也就是说操作系统已经把数据刷到磁盘上（当然磁盘也可能缓冲数据）；需要注意的是当调用fsync时只有当前的block会刷到磁盘中，要想每个block都刷到磁盘，必须在创建流时传入Sync标示。 通过分析上面的语义，可以确定hsyn涉及到两个角色client和DataNode 四、再看hbase中关于stream capabilities（hflush/hsync）功能的建议。 Controls whether HBase will check for stream capabilities (hflush/hsync). Disable this if you intend to run on LocalFileSystem, denoted by a rootdir with the 'file://' scheme, but be mindful of the NOTE below. （如果你打算在本地文件系统上运行，请禁用此选项。） WARNING: Setting this to false blinds you to potential data loss and inconsistent system state in the event of process and/or node failures. If HBase is complaining of an inability to use hsync or hflush it's most likely not a false positive. （将此设置为false会使您在节点故障时看不到潜在的数据丢失和进程中不一致的系统状态。如果HBase提示无法使用hsync或hflush，那么它很可能不是一个误报。）在我的机器上，hadoop、hbase都是配置的伪分布式，DataNode只有一个，hbase运行在LocalFileSystem（由rootdir表示），所以在配置文件（vim /usr/local/hbase/conf/hbase-site.xml）中关闭stream capabilities（hflush/hsync）功能即可。 具体要添加的配置内容如下 hbase.unsafe.stream.capability.enforce false五、查看结果 [hadoop@localhost ~]$ start-dfs.sh Starting namenodes on [localhost] Starting datanodes Starting secondary namenodes [localhost.localdomain] [hadoop@localhost ~]$ start-hbase.sh localhost: running zookeeper, logging to /usr/local/hbase/logs/hbase-hadoop-zookeeper-localhost.localdomain.out running master, logging to /usr/local/hbase/logs/hbase-hadoop-master-localhost.localdomain.out : running regionserver, logging to /usr/local/hbase/logs/hbase-hadoop-regionserver-localhost.localdomain.out [hadoop@localhost ~]$ jps 17682 HMaster 17091 SecondaryNameNode 16708 NameNode 17847 HRegionServer 18316 Jps 16845 DataNode 17533 HQuorumPeer [hadoop@localhost ~]$ hbase shell HBase Shell Use "help" to get list of supported commands. Use "exit" to quit this interactive shell. For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell Version 2.0.5, r76458dd074df17520ad451ded198cd832138e929, Mon Mar 18 00:41:49 UTC 2019 Took 0.0060 seconds hbase(main):001:0> create 'test','cf' Created table test Took 1.8818 seconds => Hbase::Table - test hbase(main):002:0> exit一切正常，问题解决！！！ ======================================================================= 参考资料： 问题的解决方法来自这里。 https://blog.csdn.net/SL_World/article/details/84952725#commentsedit HDFS中hsync方法介绍 https://www.cnblogs.com/yangjiandan/p/3540498.html hadoop官方对stream capabilities的定义 http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/StreamCapabilities.html ======================================================================= |
【本文地址】
今日新闻 |
推荐新闻 |